Gesture, Rhetoric, and Digital Storytelling

This proposal seeks to develop new theory and technology for digital interactive narratives based on cross-cultural storytelling traditions that utilize non-verbal communication to convey meaning. Digital interactive narratives are of increasing cultural relevance including: new genres of electronic literature, virtual museums and memorials, educational computer games, and computer-assisted generation of scholarly documents. We propose a new form of storytelling that changes emotional tone, theme, perspective, other subjective elements based upon gestural input via touch-screen and motion-sensitive devices. This proposal builds upon the Principal Investigator's significant body of work in developing humanistically grounded interactive narrative and artificial intelligence. The Principal Investigator has found that literary, cultural, and performance studies theories of orature (oral literature) and accounts of non-verbal communication from animation studies provide models that can be adapted to interactive narratives

Designing empowering and critical identities in social computing and gaming

This article presents theory and technology addressing the design of computing technologies for social computing and gaming: (1) to provide dynamic means of identity representation while avoiding stigmatising norms, and (2) to provide for critical reflection on stigmatising identity infrastructures found in other systems. The theory and technologies developed with these aims are encapsulated under the rubric of the Advanced Identity Representation (AIR) Project that initiated in the Imagination, Computation, and Expression Laboratory (ICE Lab) directed by the author. This work has a basis in the cognitive science foundations of categorisation and metaphor-based bias, and study of social classification infrastructures from sociology of science. Using this theoretical framework, this article provides a model to reveal a set of inadequacies of many current identity infrastructures in social computing and gaming systems for supporting the needs of collective groups of people and individuals in marginalised categories. As results, several social networking systems and games developed in the ICE Lab to empower users in the creative goal of constructing computational identities and/ or critiquing the phenomenon of stigma in these applications are presented

Computationally Modeling Narratives of Social Group Membership with the Chimeria System

Narratives are often used to form, convey, and reinforce memberships in social groups. Our system, called Chimeria, implements a model of social group membership. Here, we report upon the Chimeria Social Narrative Interface (Chimeria-SN), a component of the Chimeria system, that conveys this model to users through narrative. This component is grounded in a sociolinguistics model of conversational narrative, with some adaptations and extensions in order for it to be applied to an interactive social networking domain. One eventual goal of this work is to be able to extrapolate social group membership by analyzing narratives in social networks; this paper deals with the inverse of that problem, namely, synthesizing narratives from a model of social group membership dynamics

Identifying regional trends in avatar customization

Since virtual identities such as social media profiles and avatars have become a common venue for self-expression, it has become important to consider the ways in which existing systems embed the values of their designers. In order to design virtual identity systems that reflect the needs and preferences of diverse users, understanding how the virtual identity construction differs between groups is important. This paper presents a new methodology that leverages deep learning and differential clustering for comparative analysis of profile images, with a case study of almost 100 000 avatars from a large online community using a popular avatar creation platform. We use novelty discovery to segment the avatars, then cluster avatars by region to identify visual trends among low-and high-novelty avatars. We find that avatar customization correlates with increased social activity, and we are able to identify distinct visual trends among the U.S.-region and Japan-region profiles. Among these trends, realistic, idealistic, and creative self-representation can be distinguished. We observe that the realistic self-expression mirrors regional demographics, idealistic self-expression reflects shared mass-media tropes, and creative self-expression propagates within the communities

Comparing Clustering Approaches for Modeling Players' Values through Avatar Construction

Abstract Videogame avatars provide an expressive avenue for players to represent themselves virtually. Research has shown that these avatars, while virtual, can reveal aspects of players' identities, along with physical, social, and cultural values of the real-world. In this paper, we present an approach for modeling player values through their avatars using artificial intelligence (AI) clustering techniques. In a study with 191 participants who created avatars using our system, we provide a thorough comparison of the techniques across numerical, textual, and visual data. Our findings showed that these data structures can effectively reveal players' values and preferences, such as conforming to stereotypes of character roles using statistical attributes, modeling nuances in text descriptions of avatars, and identifying "bestexample" (prototypical) avatar appearances that players can be quantitatively shown to conform to. Our findings suggest that AI clustering approaches can be used to model players to yield insight into implicitly held values in a data-driven manner through virtual avatars

Contours of Virtual Enfreakment in Fighting Game Characters

Characters in fighting videogames1 such as Street Fighter V and Tekken7 typically reveal a phenomenon that we define as virtual enfreakment: their bodies, costumes, and fighting styles are exaggerated (1) in a manner that emphasizes perceived exoticism and (2) to enable them to be easily visually and conceptually distinguishable from one another. Here, using both quantitative and qualitative methods, including crowd-sourced surveys and analyses of game mechanics, we report on the contours of virtual enfreakment in those games. We specifically examine differences in character design across gender, national-origin, and skin-color lines. Disappointingly but not surprisingly, we find racism and sexism manifest as stark differences in character design by gender and skin color. This has strong implications because taking on the roles of these characters can have impacts on users in the physical world, e.g., performance and engagement, behavior, and understandings of others (Lim and Harrell 2015; Şengün 2015; Yee et al. 2012, Şengün et al. 2022a; Harrell and Veeragoudar Harrell 2012; Kao and Harrell 2015; Şengün 2014; Kocur et al. 2020). Although the differences are not always straightforward, female characters and darker-skinned characters (typically, characters of color) are enfreaked differently than their light-skinned male counterparts. Our results also reveal the strategic use of “unknown” as a country of origin for villainous characters. Through our mixed-methods analysis, we examine in detail how virtual enfreakment is influenced by sexism and racism, and our findings are compatible with information about the development history of the Street Fighter and Tekken franchises. However, we also find that recent characters designed in dialogue with developers from their regions of origin are some of the least enfreaked and most positively portrayed—suggesting the possibility of designing and deploying such characters for implementing anti-bias character designs within popular videos

Embracing Virtual Reality Technology with Black Adolescents to Redress Police Encounters

As Black youth face race-related stress from personal and vicarious experiences with police, practices advancing youth’s coping self-efficacy and agency are needed. We describe the pilot of a program supporting Black adolescents in creating virtual narratives detailing encounters and resolutions with police and offer preliminary observations of how this program could facilitate racial coping and emotional support. The program included four weeks consisting of both curriculum-based instruction and hands-on activities, four weeks solely focused on designing and developing students’ projects, and one week devoted to students’ final project presentations and peer feedback. We utilized a participatory design to co-create narratives with four high school students in Detroit, Michigan. We discuss how these processes can aid in the development of programs designed to reduce Black youth’s racial stress and improve youth’s coping self-efficacy. Our design and practical recommendations contribute to current literature investigating the utility of virtual narratives in redressing coping strategies for Black youth

The Arts, HCI, and Innovation Policy Discourse: Invited Panel

Although both HCI and innovation policy discourse have a STEM (science, technology, engineering, and mathematics) basis, both also include trends that incorporate the arts. The purpose of this panel is to show how HCI/arts discourse and innovation policy/arts discourse inform each other. We then discuss with the audience how innovation initiatives configure programs and roles for artists and HCI professionals working in HCI/arts

From 2,772 segments to five personas: Summarizing a diverse online audience by generating culturally adapted personas

Understanding users in the era of social media is challenging, requiring organizations to adopt novel computation-aided approaches. To exemplify such an approach, we retrieved information on millions of interactions with YouTube video content from a major Middle Eastern media outlet, to automatically generate personas that capture how different audience segments interact with thousands of individual content pieces. Then, we used qualitative data to provide additional insights into the automatically generated persona profiles. Our findings provide insights into social media usage in the Middle East and demonstrate the application of a novel methodology that generates culturally adapted personas of social media audiences, summarizing complex social analytics data into human portrayals that are easy to understand by end users in real organizations.</p